Image-based Rendering of Complex Scenes from a Multi-Camera Rig

Image-based rendering is a method to synthesize novel views from a set of given real images. This contribution describes two methods to extrapolate novel views of complex scenes with occlusions and large depth discontinuities from images of a moving uncalibrated multi-camera rig. The real camera viewpoints are calibrated from the image data and dense depth maps are estimated for each real view. Novel views are synthesized from this representation with view-dependent image-based rendering techniques at interactive rates. Since the 3D scene geometry is available in this approach, it is well suited for Mixed Reality applications where synthetic 3D objects are seamlessly embedded in the novel view.